An endoscope is an elongated tubular structure that is inserted into body cavities to examine them. The endoscope includes a telescope with an objective lens at its distal end. The telescope includes an image-forwarding system. In rigid endoscopes it may be a series of spaced-apart lenses. In flexible endoscopes it may be a bundle of tiny optical fibers assembled coherently to forward the image. This invention is applicable to both types of image forwarding systems.
At the proximal end of the image-forwarding system is an ocular lens, which creates a virtual image for direct human visualization. Often, a camera means such as a charge coupled device (CCD) chip, is mounted to the endoscope. It receives the image and produces a signal for a video display. A CCD is a semiconductor component that is used to build light-sensitive electronic devices such as cameras and image scanners. Each CCD chip consists of an array of light-sensitive photocells that produce an analog output proportional to the intensity of the incident light.
While surgeons can, and often do, look directly into the endoscope through an ocular lens, it is more common for them to use an attached video camera and observe an image on a video screen. In a surgical or diagnostic procedure, the surgeon uses the endoscope. He may tilt it, push it in, pull it out, and also rotate it around its mechanical axis. As these manipulations occur to an endoscope with an attached video camera, the camera faithfully relates what it sees, with its own upright axis displayed as the upright axis of the image on the display. This means that if the camera is rigidly fixed to the endoscope, and the endoscope-camera is moved to view a body region by, for example, rotating the instrument, the displayed image on the monitor will move proportionately and in the opposite direction to that of the endoscope camera. For example, a clockwise rotation of the endoscope-camera, or the distal end of the endoscope, through an angle of 45 degrees will still produce an upright image on the display. But in the reference frame of the user, who caused the clockwise rotation of the instrument, the image should be viewed as if it had been rotated clockwise. Since the image remains upright, it appears to the user as if there was a counterclockwise rotation of the image on the monitor through an angle of 45 degrees.
That is the very problem. When the image is displayed on the screen and the endoscope is rotated around its axis, it is as though the surgeon must tilt his head to follow it. However, the surgeon is standing up, and the rotating image is distracting to him. What he really wants to see on the screen is an image that is oriented the same as he would see it if he were inside, standing up, with the same upright orientation. Stated otherwise, he would prefer to see what he would see if he were looking directly into the endoscope, instead of viewing a screen. This is impossible when the camera is fixed to the telescope and rotates with it, while the surgeon does not.
In a conventional endoscope and camera arrangement, the camera is usually detachably and rotatably connected to the endoscope. In this arrangement the rotated image on the monitor screen can be righted by manually counter-rotating only the camera such that its orientation is upright. Alternatively, one can avoid this rotated image condition by holding the camera in its upright position and rotating only the endoscope.
Suggestions have been made to decouple the camera from the telescope so the camera can rotate independently of it, using a pendulum to seek the vertical. This seemingly sensible approach runs afoul of conditions imposed by the use of the instrument. Endoscopes are used in close quarters, and their proximal ends must be kept as small and uncluttered as possible. Physical interference with surroundings and with the surgeon's hands must be eliminated or greatly minimized. However, a pendulum to be useful must have a substantial mass and a substantial arc to work through, requiring enlargement of the instrument. Furthermore, when the endoscope is tilted, the axis of rotation of the pendulum is no longer horizontal. Now there must be bearings to support the pendulum, and the component of the force of gravity acting on the pendulum is reduced. Even worse, when the slope is very steep, a mechanical pendulum may not receive a sufficient force to seek the vertical.
Sometimes, however, there may be reasons to attach the endoscope such that it cannot rotate with respect to the camera. Or, alternatively, it may be desirable to embed the video camera within the endoscope housing. In these circumstances it is not possible to manually rotate the camera with respect to the endoscope, so some other means is necessary to right the displayed image. Furthermore, it is desirable to have this image rotation occur automatically so that, regardless of the physical orientation of the endoscope-camera in space, the displayed image of an object will always be correctly oriented with respect to the viewer's reference frame.
In addition to the rotation effects, a further perspective distortion occurs from the difference between viewing the objects directly in three-dimensions with the eyes and on a two-dimensional camera image. This perspective distortion occurs when the endoscope/camera combination views an object from a vantage point that is above (or below) and to the side, relative to the surgeon's direct “line-of-sight.” The vanishing point of the perspective view is on the side of the rendered object furthest from the endoscope's vantage point. This results in objects closest to the endoscope end appearing disproportionately large and also results in horizontal lines appearing tilted in the display.
U.S. Patent Application Ser. No. 60/155,850 of Chatenever and U.S. Pat. No. 6,097,423 disclose a device for correcting for the rotation of the endoscope's distal end. That invention uses a single accelerometer to determine the angular displacement of the endoscope using the direction of gravity, as sensed with the accelerometer, for a vertical reference and as described in the '423 patent rotates a CCD image sensor aligned with the optical axis of the endoscope so as to maintain a desired orientation of a display of the image on a monitor.
U.S. Pat. No. 5,881,321 to Kivolowitz, Mar. 9, 1999, discloses a system for using absolute position of a hand-held camera by use of inertial sensors incorporated into the structure of the camera to detect the movement of the camera along three orthogonal axes, as well as angular rotation around the three axes. This device uses a wireless communication device for transmitting the position data and remote processing to alter the generation of images. The wireless communication approach, while appropriate for the larger video or motion picture camera contemplated therein, adds batteries and considerable circuitry and therefore size which is unavailable in the tight quarters required in an endoscope. Additionally, no provision is disclosed for mechanical alignment of the image prior to the processing for display.